Video processing device, video processing method, and program

ABSTRACT

A video processing device includes a video accumulating unit configured to accumulate video data obtained by capturing video of a specific area, event occurrence data indicating an event that has occurred in a video image corresponding to the video data, and object specifying data for specifying an object relating to the event, a video processing unit configured to apply a predetermined effect to the video data accumulated in the video accumulating unit in accordance with the event occurrence data and the object specifying data that are added to the video data accumulated in the video accumulating unit to produce a modified video image, and a video output unit configured to output the modified video image to which the effect has been applied by the video processing unit.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority from Japanese Patent ApplicationNo. JP 2009-024302 filed in the Japanese Patent Office on Feb. 4, 2009,the entire content of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a video processing device and methodsuitable for processing video of, for example, a recorded sports game,and to a program using the method.

2. Description of the Related Art

When a live broadcast game of sport is recorded, scenes of the game aregenerally captured using a plurality of cameras, and editing operationsare performed such as appropriately switching between the cameras inaccordance with the progress of the game. Finally, video content isproduced.

When the video content is reproduced, a guide image representing detailssuch as the score and the names of players may be displayed, as desired,in such a manner that the guide image is superposed on a reproducedvideo image.

Further editing operations such as reproduction of an impressive sceneof actions in a game, for example, scoring a goal in a soccer game,again immediately after the same scene or insertion of shots of thescene at different angles can also be performed and finally videocontent is produced. Editors can also apply special effects to videofiles.

Recently, detailed information about the progress of sports games hasbeen electronically created using computer devices and has beenpresented or accumulated via the Internet or the like. Such informationabout sports games is called stats.

Stats is information that is manually entered by a stats informationinput operator using a computer device or a dedicated input device in astadium where an actual sports game is taking place while the operatoris watching the game, such as an ongoing status of the game and thenames of players. This information is hereinafter referred to as“stats”, “stats information”, or “stats data”. Stats information allowsa viewer to readily obtain the details of, for example, a soccer game,such as which player tried a shot on goal and which player scored agoal.

Japanese Unexamined Patent Application Publication No. 2007-102503describes an example of the generation of motion data reflecting themotion of players in a video game, which is necessary to execute a videosports game using a gaming device.

SUMMARY OF THE INVENTION

As described above, video content of a recorded sports game is generallyedited by an editor to produce video content. The production of videocontent may involve much labor for editing operations.

In addition, the above-described stats information about a sports matchexists irrespective of recorded video of the sports match, and is notgenerally associated with the recorded video of the sports match. When asport match is broadcast live, the latest score or a brief progressreport may be transmitted via data broadcasting. However, detailedinformation associated with video content, for example, informationabout the actions of players, is not generally available.

As disclosed in Japanese Unexamined Patent Application Publication No.2007-102503, a technique for generating motion data reflecting themotion of players in a sports game is used in virtual video sports gameson video gaming devices. However, such a technique is not generallyapplied to video of recorded real sports games. Much time and effort isdevoted to creating a program for playing a sports game on a videogaming device, and such a program is not practically applicable toactual sports games.

It is therefore desirable to associate video of a sports match withdetailed information about the match to facilitate easy and high-qualitydisplay of video.

In an embodiment of the present invention, a video accumulating unitaccumulates video data obtained by capturing video of a specific range,event occurrence data indicating an event that has occurred in a videoimage corresponding to the video data, and object specifying data forspecifying an object relating to the event.

A predetermined effect is applied to the video data accumulated in thevideo accumulating unit on the basis of the event occurrence data andobject specifying data added to the video data accumulated in the videoaccumulating unit to produce a modified video image. Then, the videodata image to which the effect has been applied is output.

Therefore, the output video data is displayed, thereby allowing, forexample, when the video data is obtained by capturing video of a sportsgame, a viewer to obtain an ongoing status of the sports game, theactions of players, and the like.

According to an embodiment of the present invention, an event that hasoccurred in a captured video image, and an object involved in the eventcan be specified to produce a video image that has been automaticallyedited.

For example, when a video sequence of a sports match such as a soccergame is obtained, the time when an event such as a scoring scene duringthe match occurred and a player involved in the event can be specifiedon the basis of additional data. A video image that has been subjectedto editing processes such as enlarging the specified player or depictingthe movement of the specified player can be displayed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example configuration of asystem according to an exemplary embodiment of the present invention;

FIG. 2 is a block diagram illustrating an example of a full statsgenerating device according to an embodiment of the present invention;

FIG. 3 is a diagram illustrating a captured video image representing theentire area of a field according to an embodiment of the presentinvention;

FIG. 4 is a diagram illustrating an example of stats data according toan embodiment of the present invention;

FIG. 5 is a diagram illustrating an example of data format (example ofoverview format) according to an embodiment of the present invention;

FIG. 6 is a diagram illustrating an example of data format (example offixed position data) according to an embodiment of the presentinvention;

FIG. 7 is a diagram illustrating an example of data format (example oflive position data) according to an embodiment of the present invention;

FIG. 8 is a diagram illustrating an example of data format (example offixed event data) according to an embodiment of the present invention;

FIG. 9 is a diagram illustrating an example of data format (example oflive event data) according to an embodiment of the present invention;

FIG. 10 is a diagram illustrating an example of data format (example offile header) according to an embodiment of the present invention;

FIG. 11 is a diagram illustrating an example of data format (example ofball position file) according to an embodiment of the present invention;

FIG. 12 is a diagram illustrating an example of data format (example ofplayer position file) according to an embodiment of the presentinvention;

FIG. 13 is a diagram illustrating an example of data format (example ofplayer event file) according to an embodiment of the present invention;

FIG. 14 is a diagram illustrating an example of data format (example offile trailer) according to an embodiment of the present invention;

FIG. 15 is a flowchart illustrating an exemplary video editing processaccording to an embodiment of the present invention;

FIG. 16 is a flowchart illustrating an exemplary edited videosynchronization process according to an embodiment of the presentinvention;

FIG. 17 is a diagram illustrating an example of a video image andmeta-data according to an embodiment of the present invention;

FIG. 18 is a diagram illustrating an example of a video image that isprocessed under the condition of FIG. 17, in which a portion of awide-angle video image is zoomed and reproduced;

FIG. 19 is a diagram illustrating another example of a video image andmeta-data according to an embodiment of the present invention;

FIG. 20 is a diagram illustrating an example of a video image that isprocessed under the condition of FIG. 19;

FIG. 21 is a diagram illustrating an example (first example) of areproduced video image according to an embodiment of the presentinvention;

FIGS. 22A and 22B are diagrams illustrating examples (second examples)of reproduced video images according to an embodiment of the presentinvention;

FIGS. 23A and 23B are diagrams illustrating examples (third examples) ofreproduced video images according to an embodiment of the presentinvention;

FIG. 24 is a diagram illustrating an example (fourth example) of areproduced video image according to an embodiment of the presentinvention;

FIG. 25 is a diagram illustrating an example (fifth example) of areproduced video image according to an embodiment of the presentinvention; and

FIG. 26 is a diagram illustrating an example (sixth example) of areproduced video image according to an embodiment of the presentinvention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

An exemplary embodiment of the present invention will be describedhereinafter with reference to the drawings in the following order:

(1) Example Configuration of Overall System: FIG. 1

(2) Example Configuration of Full Stats Generating Device: FIG. 2

(3) Example of Stats Information: FIG. 4

(4) Examples of Data Structure: FIGS. 5 to 14

(5) Exemplary Video Editing Process: FIGS. 15 and 16

(6) Example of Association between Video Image and Meta-Data: FIGS. 17to 20

(7) Example of Reproduced Video Image: FIGS. 21, 22A, 22B, 23A, and 23B

(8) Other Examples of Reproduced Video Images: FIGS. 24 to 26

(1) Example Configuration of Overall System: FIG. 1

First, an example configuration of an overall system according to anexemplary embodiment of the present invention (hereinafter referred toas the present embodiment) will be described with reference to FIG. 1.In the present embodiment, a system configured to capture video of asoccer game using cameras in a soccer stadium and to accumulate thecaptured video of the soccer game and other data is used by way ofexample.

In the present embodiment, a plurality of cameras 11 and 12 are placedin a stadium 10 where a sports game takes place, and video of the gameis captured using the cameras 11 and 12, and video data of the capturedvideo is supplied to an editing device 13. At least one camera is usedto capture video of the entire area of a field where the game takesplace in the stadium 10.

For example, as illustrated in FIG. 3, video data captured using acamera located at a fixed position so as to capture a video imagerepresenting the entire area of a field in a stadium is obtained. In theexample of FIG. 3, it is assumed that video images captured using twocameras placed side by side are connected to form a video imagerepresenting the entire area of the field in the stadium.

The editing device 13 performs editing operations on a video sequence ofimages of the game, such as selection of a video image including adesired scene captured using a camera. The editing device 13 may performan editing operation at the same time as the capture operation performedin the stadium 10. A video image representing the entire area of thefield may be constantly output irrespective of an edited video image.

An input device 14 is also located in the stadium 10. An operator entersstats information, such as the progress of the game, using the inputdevice 14. The input device 14 may be configured by, for example,installing dedicated software into a personal computer device.

The input device 14 is operated by an operator to enter the detailedprogress of the game as well as data such as the names of teams takingpart in the soccer game, the names of players, and the names ofreferees. The detailed progress of the game may include, in addition tobasic data such as goal scoring, information about the occurrence ofevents such as a shot on goal, a foul, and a corner kick, together withthe names of players involved in the respective events. Informationabout the rough positions of the players on the field may also beentered if available. The information about the progress of the game mayfurther include the time elapsed since the start of the game or anabsolute time.

The above processes may be performed in real time during the game inprogress.

The video data edited by the editing device 13 and the stats informationentered using the input device 14 are sent to a full stats generatingdevice 20. The operation of the full stats generating device 20 isbasically performed after the end of the soccer game which is played inthe stadium 10.

The full stats generating device 20 is connected to an input device 21,and receives detailed stats information. The full stats generatingdevice 20 processes the stats information entered using the input device14 to produce complete stats information (full stats information). Ifthe stats information entered using the input device 14 contains anerror, the error is corrected. The input device 21 may also beconfigured using a computer device.

Examples of the stats information entered using the input device 21include detailed data that is difficult to enter in real time, such asdata on the positions of the players or referees in the field, data onthe orientations of the faces of the players or referees, and data onthe position of the ball. The above data is generally entered manuallyby an operator, or may be automatically generated on the basis ofcaptured video of the entire area of the field in the stadium 10.

The full stats generating device 20 is also connected to a videoprocessing device 22, and the video processing device 22 edits a videosequence of images of the game. The editing operation of the videoprocessing device 22 is performed automatically to some extent using thestats information created by the full stats generating device 20.

The stats information created by the full stats generating device 20 andedited video data created by the video processing device 22 are suppliedto and accumulated in a server 30. In the server 30, video data andstats information about a single match are accumulated as data on asingle piece of video content. When there is audio data attached to thevideo data, the audio data is also accumulated.

Video content of a soccer game, which is accumulated in the server 30,is delivered to, for example, terminal devices 31, 32, and 33 that haveaccessed it together with the accompanying stats information.Alternatively, the video content is stored in a certain storage mediumand is distributed to users as desired.

(2) Example Configuration of Full Stats Generating Device: FIG. 2

Next, an example configuration of the full stats generating device 20and peripheral devices will be described with reference to FIG. 2.

A live video capture unit 101 receives video data transmitted from theediting device 13 illustrated in FIG. 1. A video sequence captured bythe live video capture unit 101 is supplied to a video processing unit102 and is accumulated in a buffer 103 as desired.

Data input using a sports game event input unit 104 is sent to an eventdata transmitting function unit 105. The sports game event input unit104 and the event data transmitting function unit 105 correspond to theinput device 21 illustrated in FIG. 1.

Event data output from the event data transmitting function unit 105 isreceived by an event data receiving function unit 106, and is sent to anevent handler unit 107. The event handler unit 107 sends a video editinginstruction to a video cutting unit 108 on the basis of the event data.

In accordance with the instruction from the event handler unit 107, thevideo cutting unit 108 performs an editing process such as extractionand enlargement of a video image of a specific portion in the videosequence supplied to the video processing unit 102. Video data editedusing the video processing unit 102 is accumulated in a storage unit110. The event data (stats information) received by the event datareceiving function unit 106 is also accumulated in the storage unit 110.The video data accumulated in the storage unit 110 may further be editedby the video processing device 22 illustrated in FIG. 1. In FIG. 2, onlythe processing of event data as stats information is illustrated. Otherdata such as position information about individual players may also beentered and processed with a processing configuration similar to thatdescribed above.

The edited video data and the stats information are transmitted to theserver 30 illustrated in FIG. 1.

(3) Example of Stats Information: FIG. 4

Next, an overview of stats information that is added to video data inthe system according to the present embodiment will be described withreference to FIG. 4.

As illustrated in FIG. 4, from the beginning of a soccer match until theend of the match, event data and position information data regarding theplayers, the referees, and the ball are added.

As illustrated in FIG. 4, the event data contains information for eachevent that has occurred during the match, such as a kickoff, a pass, atouch, or a shot on goal, including the time at which the eventoccurred, a number for specifying the event, and which player wasinvolved in the event. The position information data may be dataindicating the coordinate position of all the players on the field, thereferees, and the ball at predetermined time intervals such as persecond. The event data and the position data are statistically processedand analyzed, thereby providing an analysis of the details per match,such as the number of shots on goal and the number of goals scored. Inthe example of FIG. 4, the success rate of shots with the right foot andthe success rate of shots with the left foot are calculated. It is to beunderstood that the data items illustrated in FIG. 4 are merely examplesand other detailed analysis can be made using a detailed data structuredescribed below.

(4) Examples of Data Structure: FIGS. 5 to 14

Next, the data structure of the stats information generated by the fullstats generating device 20 will be described with reference to FIGS. 5to 13.

First, an overview of the data structure of the stats information willbe described with reference to FIGS. 5 to 9.

In the present embodiment, as illustrated in FIG. 5, the statsinformation has a data format including a header section, a bodysection, and a trailer section. The body section contains actual datadescribed below.

FIGS. 6 to 9 illustrate examples of the body section.

FIG. 6 illustrates fixed position data. The fixed position data may bedata indicating a fixed position of an individual player on the field ina soccer match. That is, data on a position that is assigned to eachplayer, such as the center forward, is set and arranged player-by-playerfor each frame. The fixed position data may be data that is basicallysufficient to be added once to data for each match unless a player isreplaced.

FIG. 7 illustrates live position data. The live position data may bedata indicating the coordinate position of all the players on the field,the referees, and the ball at each time step. Here, as illustrated inFIG. 7, a frame number of a video sequence is indicated per unit of liveposition data, and the positions of the players and the like in thevideo sequence at the corresponding frame number are indicated. In themost detailed data, therefore, representations per frame are presented.Representations per second may be presented in order to reduce the dataamount.

FIG. 8 illustrates fixed event data. The fixed event data includes, foreach event, the frame number at which the event has occurred and dataindicating the details of the event that has occurred. The fixed eventdata may be data on fixed events that regularly occur during a match,such as the start of the match, the end of the match, and stoppage.

FIG. 9 illustrates live event data. The live event data includes, foreach event, the frame number at which the event has occurred and dataindicating the details of the event that has occurred. Live events maybe events that occur at any time during a match, such as a pass, atouch, a shot on goal, and a corner kick, and data regarding playersinvolved in the events is added in detailed data. The live event datamay be data that is continuously generated during a match. Asillustrated in FIG. 9, the header and trailer are added to every certainamount of data to form a packet.

Next, a detailed example data configuration of data items will bedescribed with reference to FIGS. 10 to 14. In FIGS. 10 to 14, the dataitems are represented in units of a predetermined number of bytes.

FIG. 10 illustrates an example of a file header. In the file header,data items indicating a data structure, such as the format type, theformat version, and the frame rate, are added at the beginning, followedby the ID of a soccer competition (“competition ID”), the ID of a soccermatch (“match ID”), the ID of the home team (“home team ID”), and the IDof the away team (“away team ID”). Other data items such as thehorizontal and vertical dimensions of the field (X-direction field sizeand Y-direction field size) and the number of players are also added.

As illustrated in FIG. 10, the time of the start of the first half, thetime of the end of the first half, the time of the start of the secondhalf, the time of the end of the second half, the time of the start ofthe first half of extra-time, the time of the start of a penaltyshootout (“PK start”), and other data items are included. In theillustrated example, the time scale indicator is based on the absolutetime or UTC (Universal Time Coordinated) time. A detailed structure ofthe information about each time stamp is illustrated in the right partof FIG. 10.

FIG. 11 illustrates an example configuration of a ball position file.

The position of the ball during a match is represented in terms of theUTC time, the frame number of video, the X and Y coordinate positions onthe field, and the height-direction (Z) coordinate position.

FIG. 12 illustrates an example configuration of a player position file.

The player position file describes, for each player, the X and Ycoordinate positions on the field and the height-direction (Z)coordinate position. The position of the face of each player is furtherrepresented in terms of the X and Y coordinate positions on the fieldand the height-direction (Z) coordinate position. The face data of eachplayer may further include data on the orientation of the face of theplayer.

The player position file further includes detailed data regarding theactions of an individual player, such as a period in which the playermoves the foot.

FIG. 13 illustrates an example configuration of a player event file.

The player event file contains data on the details of an event that isplayed by an individual player during a match. The player event fileincludes a frame number, a player ID, a team ID, a shirt number(“uniform number”), and event data representing the details of an event.

FIG. 14 illustrates the file trailer which describes the total number ofevents and the number of frames of video.

(5) Exemplary Video Editing Process: FIGS. 15 and 16

FIGS. 15 and 16 are flowcharts illustrating exemplary processes forediting a video sequence on the basis of stats information obtained inthe manner described above. FIG. 15 illustrates an example in which aneffect that is based on meta-data is superposed when a video image isreproduced, and FIG. 16 illustrates an example in which an effect isgenerated from meta-data and is superposed when a video image isreproduced. The processing steps in the flowcharts may be automaticallyperformed in, for example, the video processing device 22 illustrated inFIG. 1. Alternatively, for example, an indication for selecting anediting operation on the basis of the stats information may be displayedon a screen that is viewed by an editing operator, and an operator mayselect an editing operation and determine whether or not the selectedediting operation is to be executed. In the flowcharts, the term“meta-information” and “meta-data” correspond to the stats informationdescribed above.

First, a wide-angle captured video image representing the entire area ofthe field (FIG. 3) is reproduced using the video processing device 22(step S11), and it is determined whether or not meta-information isassociated with the current reproduction time (step S12). Whenmeta-information (here, live event data in the stats information) is notassociated, the process returns to step S11.

If it is determined in step S12 that meta-information (live event datain the stats information) is associated, an area where the event isoccurring in the wide-angle video image is determined on the basis ofthe coordinate position of the player or players or the ball, which isindicated by the meta-information (step S13). An editing operation(effect) to be applied to the video image is also determined inaccordance with the type of the event.

Then, the video image currently being reproduced is edited so as toreflect the determined area and effect (step S14), and edited video datais extracted. Then, the process returns to step S11. Specific examplesof the edited video image are described below.

FIG. 16 illustrates an exemplary process for synchronizing the extractededited video data with the original video image.

First, meta-data is analyzed and an editing operation (effect) isdetermined (step S21). Then, it is determined whether or not video datahas been created by performing the editing operation (effect) (stepS22). The process waits for desired video data to have been created.When desired video data is created, a time synchronization process forsynchronizing the reproduction time of the wide-angle video imagerepresenting the entire area of the field with the reproduction time ofthe created video image is performed (step S23). By performing the timesynchronization process, video content in which the wide-angle videoimage and the edited video image are reproduced at the same time iscreated (step S24). Then, the process returns to step S21.

(6) Example of Association between Video Image and Meta-Data: FIGS. 17to 20

Next, an example of the association between video data of a wide-anglevideo image and meta-data will be described with reference to FIGS. 17to 20.

For example, as illustrated in FIG. 17, it as assumed that meta-dataindicates that a player at position “A-5” made a shot on goal at acertain time.

In this case, as illustrated in FIG. 18, a portion including the playeris extracted from the wide-angle video image, and edited video data inwhich the extracted video image portion is enlarged is created. Theenlarged video image portion is presented together with the wide-anglevideo image.

For example, as illustrated in FIG. 19, it is further assumed thatmeta-data indicates that a certain player passed the ball to anotherplayer at the timing of a certain frame number.

In this case, as illustrated in FIG. 20, a video image in which thelocus of the ball is drawn is created, and the video image showing thelocus of the ball is displayed in a manner superposed on the wide-anglevideo image.

In the illustrated examples, only the wide-angle video imagerepresenting the entire area of the field is obtained by way of example.When a video image captured at a different angle can be obtained, thevideo image captured at the different angle can be used to create editedvideo data.

(7) Example of Reproduced Video Image: FIGS. 21, 22A, 22B, 23A, and 23B

FIGS. 21, 22A, 22B, 23A, and 23B are diagrams illustrating more specificexamples of edited video images.

In the example illustrated in FIG. 21, a wide-angle captured video imagerepresenting the entire area of the field is located in a lower portionof the screen, and edited video images that are created on the basis ofmeta-data are located in a portion upper than the wide-angle videoimage, where scores and a list of names of players are shown inappropriate portions. According to the present embodiment, the abovevideo images can be automatically created through the use of meta-data.

FIGS. 22A, 22B, 23A, and 238 illustrate examples of video images otherthan a wide-angle video image representing the entire area of the field.In the examples, video images captured at different angles are extractedon the basis of meta-data and edited video images are created. Inaddition, information obtained by analyzing the meta-data is alsosuperposed using characters and any other suitable items. The videoimages illustrated in FIGS. 22A, 22B, 23A, and 23B may be presented andreproduced together with the wide-angle video image representing theentire area of the field to create video content.

(8) Other Examples of Reproduced Video Images: FIGS. 24 to

FIGS. 24 to 26 illustrate examples of other display forms using acaptured video image representing a soccer match and stats informationabout the soccer match according to the present embodiment.

In the example illustrated in FIG. 24, a video image of a relativelylarge area that is centered on the position of the ball within awide-angle video image representing the entire area of the field islocated in an upper left portion of the screen. Therelatively-large-area video image may be a video image extracted fromthe wide-angle video image representing the entire area of the field ormay be a video image captured using a different camera.

In an upper right portion of the screen, a specific player is displayedin an enlarged manner. The player to be displayed in an enlarged mannermay be, for example, a player specified in advance in accordance with auser operation, or a player who is near the ball may be automaticallyselected and displayed in an enlarged manner. Alternatively, in a casewhere the position of the ball can be specified, an area that iscentered on the position of the ball may be enlarged and displayed. Inthe illustrated example, the scale factor can be selected in accordancewith a user operation. A scrollbar for changing the scale factor may bedisplayed. In the example illustrated in FIG. 24, a scale factor of 200%has been set. The scale factor may be a scale factor by which the videoimage representing the entire area, which is located in the upper leftportion, is enlarged. In the wide-angle video image, a range withinwhich corresponding to the enlarged video image is included isindicated.

When a specific player is displayed in an enlarged manner, for example,in a case where the orientation of the face of the player can bedetermined on the basis of stats information and in a case where theplayer can be captured using cameras from a plurality of directions, avideo image captured using a camera from a direction in which the faceof the player can be clearly identified may be enlarged and displayed.

In the example illustrated in FIG. 24, furthermore, a corner kick duringa soccer match is made. In the upper-left video image representing theentire area, the pattern of previous corner kicks, such as directions inwhich the ball was kicked, may be represented in terms of probability,or previous success rates may be represented using characters ornumerals.

The data on the numerals and the like may be represented per player orper match.

In the example illustrated in FIG. 24, furthermore, a graphicalrepresentation is displayed in a lower portion of the screen, where thepositions of individual players throughout a soccer match are indicatedby figures. In the example illustrated in FIG. 24, the entire area ofthe field is viewed from the long edge thereof. The positions ofindividual players are determined on the basis of the wide-angle videoimage representing the entire area of the field or the statsinformation, namely, information about the actions of the players, andfigures that simulates the players are shown in the graphicalrepresentation of the field. Each of the players is represented usingthe corresponding figure with the team name or the number. The positionsof the ball and the referees are also represented using figures.

As with options indicated in a left edge portion of FIG. 24, thedirection in which the field is displayed can be selected in accordancewith a user operation. For example, one of a view from the long edge ofthe field (wide view), which is illustrated in FIG. 24, a view frombehind the goal of the home team, and a view from behind the goal of theaway team can be selected. An individual player of the home team, anindividual player of the opposing team (away team), or an individualreferee can be selected as an object to be displayed. In the example ofFIG. 24, all the objects are displayed.

The example illustrated in FIG. 25 takes the same display form as theexample of FIG. 24, and a video image captured at a different time inthe same match as that illustrated in FIG. 24. As can be seen fromcomparison between FIGS. 24 and 25, the positions of the players in thegraphical representation located in the lower portion of the screen havebeen changed in accordance with the movement of the players during thematch.

FIG. 26 illustrates an example of a direction in which the graphicalrepresentation located in the lower portion of the screen is displayed,which is different from that illustrated in FIG. 24. In the illustratedexample, the positions of the individual players are represented usingfigures and characters, when viewed from behind the goal of the awayteam.

According to the present embodiment, therefore, a video sequence ofcaptured images of a soccer match and stats information about the soccermatch are associated in terms of frame number and absolute timeinformation, and a video image of a specific area is extracted from thevideo sequence on the basis of the stats information and is edited.Therefore, the video sequence of captured images of the match isintegrated with meta-data such as the stats information, and isautomatically edited. In this case, the editing area is determined byreferring to the meta-data, without detecting an object in the videosequence using image recognition or the like. Therefore, advantageously,high-quality processing can be performed with reduced load for videoediting processing.

While in the embodiment described above, a video sequence of images of asoccer match is used by way of example, a video sequence of images ofany other sports match can also be used.

In the embodiment described above, video data, data on an event that hasoccurred in a match, and data for specifying an object (such as aplayer) in a video sequence of images, which is involved in the event,are linked in terms of frame number of the video sequence. The abovedata items may be linked using any other data. In the embodimentdescribed above, since UTC data serving as absolute time data is added,the above data items may be linked using the UTC data. Alternatively,the above data items may be linked using the absolute elapsed time fromthe time of the start of the match or the number of elapsed frames.

The devices illustrated in FIG. 1 may also be configured byincorporating a program for executing the processes described in theembodiment into a computer device (information processing device) sothat similar devices can be achieved.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. A video processing device comprising: a video accumulating unitconfigured to accumulate video data obtained by capturing video of aspecific area, event occurrence data indicating an event that hasoccurred in a video image corresponding to the video data, and objectspecifying data for specifying an object relating to the event; a videoprocessing unit configured to apply a predetermined effect to the videodata accumulated in the video accumulating unit in accordance with theevent occurrence data and the object specifying data that are added tothe video data accumulated in the video accumulating unit to produce amodified video image; and a video output unit configured to output themodified video image to which the effect has been applied by the videoprocessing unit, wherein the modified video image produced by applyingthe predetermined effect using the video processing unit is a videoimage obtained by extracting a video image of a specific objectspecified by the object specifying data from the video data accumulatedin the video accumulating unit so as to obtain an extracted video imagewhich is displayable on a plurality of display devices, and wherein whenthe extracted video image is displayed, at least a portion thereof isenlargeable to a size selected by a user of each display device by useof a scale factor changing image which is simultaneously displayed. 2.The video processing device according to claim 1, wherein the video dataaccumulated in the video accumulating unit is video data of a sportsgame that is recorded, wherein the object specifying data includes dataon a registered position of a player who is participating in the sportsgame, and data on a position of movement of the player in the sportsgame in progress, and wherein the modified video image produced byapplying the predetermined effect using the video processing unit is avideo image representing a position or movement of a specific player whois participating in the sports game.
 3. The video processing deviceaccording to claim 2, wherein the video image representing a position ormovement of a specific player who is participating in the sports game isa video image in which positions of some or all players on a field wherethe game is taking place are represented with signs or names.
 4. Thevideo processing device according to claim 3, wherein the video image inwhich positions of some or all players on a field where the game istaking place are represented with signs or names is a graphical videoimage that is different from a video image of the sports game that isrecorded.
 5. A video processing method comprising the steps of:accumulating video data obtained by capturing video of a specific area,event occurrence data indicating an event that has occurred in a videoimage corresponding to the video data, and object specifying data forspecifying an object relating to the event; and applying a predeterminedeffect to the video data accumulated in the step of accumulating inaccordance with the event occurrence data and the object specifying datathat are added to the video data accumulated in the step of accumulatingto produce a modified video image, wherein the modified video imageproduced by the step of applying the predetermined effect is a videoimage obtained by extracting a video image of a specific objectspecified by the object specifying data from the video data accumulatedin the step of accumulating so as to obtain an extracted video imagewhich is displayable on a plurality of display devices, and wherein whenthe extracted video image is displayed, at least a portion thereof isenlargeable to a size selected by a user of each display device by useof a scale factor changing image which is simultaneously displayed.
 6. Anon-transitory computer-readable medium having stored thereon a programfor causing an information processing device to execute the steps of:accumulating video data obtained by capturing video of a specific area,event occurrence data indicating an event that has occurred in a videoimage corresponding to the video data, and object specifying data forspecifying an object relating to the event; and applying a predeterminedeffect to the video data accumulated in the step of accumulating inaccordance with the event occurrence data and the object specifying datathat are added to the video data accumulated in the step of accumulatingto produce a modified video image, wherein the modified video imageproduced by the step of applying the predetermined effect is a videoimage obtained by extracting a video image of a specific objectspecified by the object specifying data from the video data accumulatedin the step of accumulating so as to obtain an extracted video imagewhich is displayable on a plurality of display devices, and wherein whenthe extracted video image is displayed, at least a portion thereof isenlargeable to a size selected by a user of each display device by useof a scale factor changing image which is simultaneously displayed.